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Abstract 


Higher education has been deeply involved in the process of globalisation and has 
experienced consequent changes. In order to successfully operate in the globalised 
higher education landscape, universities worldwide continue to foster a 
commitment to internationalisation. As internationalisation has become a central 
concern of universities, more sophisticated information and useful tools are needed 
to help with mapping and measuring the phenomenon. This article examines how 
internationalisation is being measured in different universities in Australia and 
China. Through in-depth interviews with university leaders and head of office, the 
study presents an analysis of the current practice in 15 selected universities in 
relation to measuring the degree of internationalisation. The findings of the study 
shed light on the importance of the assessment of internationalisation as well as the 
obstacles impeding the development of effective measures for it. 


Introduction 


Higher education has been deeply involved in the process of globalisation and has experienced 
consequent changes. In the increasingly globalised environment in which every university is more 
visible to the other, it is no longer possible for individual universities to seal themselves off from 
global impacts. A new global higher education landscape is emerging and, within it, the distribution of 
capacities and resources between universities in many respects determines their global position and 
potential (Marginson & van der Wende, 2007). The impacts that globalisation is having on higher 
education largely provide the external impetus for accelerated institutional internationalisation. In 
order to operate successfully in the current environment, universities must continue to foster a 
commitment to internationalisation and make significant efforts to integrate the international 
dimension into key areas of university operation. As internationalisation has now moved from the 
margins of institutional interests to the very core, more sophisticated information and tools are needed 
to better monitor and measure this phenomenon. This paper examines how the performance of 
internationalisation is being monitored and measured in flagship universities in both Australia and 
China. It also reveals the barriers to developing effective measurement of university 
internationalisation. In this study, the term ‘internationalisation’ is used in its broadest sense and it 
includes internationalising the student body, faculty, curriculum, research, governance and 
engagement. The details will be discussed later in the paper when considering measures of 
internationalisation. In this regard, Knight’s (1994, p. 7) definition of a ‘process of integrating an 
international or intercultural dimension into teaching, research, and service functions of the institution’ 
aligns with the use of the term ‘internationalisation’ in the present study. 


Urgent demand for measurements of university internationalisation 


While internationalisation used to be an aggregation of dispersed activities within higher education 
institutions, it has evolved into a comprehensive strategy that should be approached in a holistic way. 
This shift adds to the complexity of university internationalisation and creates a need for better data. 
The challenge is not just about more internationalisation, but better internationalisation. Three reasons 
explain the urgent demand for measurements of university internationalisation. First, universities 
require reliable information in order to monitor and assess their performance, and to avoid expressing 
themselves vaguely about their internationalisation status; with the later, either what is meant or who 
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makes these assessments is explicit (Knight, 2008; Stier & Borjesson, 2010). Precise and relevant 
measures of achievements will help provide information to reduce the vagueness in university’s self- 
representation. They may also contribute to make a distinction between strategic aspiration and 
strategic reality and to recognise that, for some universities, there is a ‘gap’ between the two. This 
information enables universities to identify their areas of strength and weaknesses and then initiate 
improvement (Knight, 2008; Maringe & Foskett, 2010). 


Second, because of the increased global competition and the importance of rankings, institutions need 
data to help profile themselves and make comparisons with peers in terms of internationalisation 
(Beerkens et al., 2010). The international ranking is a widely debated example of how measurements 
have influenced university management and operation in a way that differs from the past (Marginson, 
2011). For example, universities need indicators to show the international impact of their research and 
their popularity with international students. Third, the emergence of an accountability culture in higher 
education based on evaluations has also pushed the agenda (Beerkens et al., 2010). Information needs 
to be provided to help inform students, academics, and other stakeholders to what extent an institution 
is internationalised. The need to provide the public with reliable information is not only a matter of 
reputation but also reflects an increased demand for transparency. As internationalisation requires a 
substantial commitment of institutional resources, and perhaps an opportunity cost in some other 
areas, ‘knowing objectively and in measurable terms that it produces value in the intended directions 
becomes critical in garnering support beyond rhetoric (Hudzik & Stohl, 2009, p. 12).’ 


Studies on developing measurements for university internationalisation 


Over recent decades, research involving the development of instruments for measuring university 
internationalisation has gained increasing attention. The first endeavour to assist universities in 
assessing and improving the quality of their internationalisation activities was the Jnternational 
Quality Review Programme (IQRP) developed by the Institutional Management in Higher Education 
(IMHE), OECD together with the Academic Cooperation Association (ACA) (Knight & de Wit, 
1999). Subsequently, there has been a considerable growth in the number of studies and projects that 
attempted to develop measurements of internationalisation in decade thereafter. Not surprisingly, in 
the beginning the studies were conducted in countries such as the U.S. (e.g. Green, 2005; Horn, 
Hendel, & Fry, 2007), U.K. (e.g. Ayoubi & Massoud, 2007) and Australia (see Krause, Coates, & 
James, 2005), where internationalisation gained more importance due to the increasing flow of foreign 
students. Attempts were also made in European countries such as France (e.g. Echevin & Ray, 2002), 
the Netherlands (e.g. de Wit, 2009) and Germany (e.g. DAAD, 2010). In addition, there has also been 
an increasing interest in assessing internationalisation performance in East Asia, particularly in Japan 
(see the study of Furushiro, 2006), Taiwan (see Chin & Ching, 2009), and Mainland China (e.g. Chen, 
Zeng, Wen, Weng, & Yu, 2009). 


These studies imply that the less tangible concept of internationalisation can be constructed by 
identifying the key domains of internationalisation practice that can be measured. In these studies, 
either a set of indicators or rubrics were developed to help capture different components of university 
internationalisation. For example, Paige’s (2005) study proposed 80 indicators to measure ten 
elements of university internationalisation. Krause et al. (2005) analysed 38 Australian institutions’ 
web pages and developed 66 indicators to capture 17 components of internationalisation. Osaka 
University in Japan launched a project to develop evaluation criteria for the internationalisation of 
Japanese universities, using an ‘a la carte menu’ of rubrics to reflect 23 elements. The study that was 
conducted by Brandenburg and Federkeil (2007) developed indicators to assess 20 thematic areas of 
university internationalisation, which were then clustered into three main dimensions. By 
crosschecking the identified components in different studies, it confirms the existence of common 
elements of university internationalisation. Specifically, 16 measurable components have been 
acknowledged in most studies as indicative of the extent to which an institution has engaged with 
internationalisation. These components can be grouped to reflect six key dimensions of university 
internationalisation, as shown in the figure below. 
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Table 1: Measureable components of university internationalisation 


Dimension Component 


Governance & organisational support Human resources for international activities 
Financial support for internationalisation 


International presence 


Student International students 
Mobility of students 
Faculty International profile of the faculty team 


International experience of faculty 


Curriculum Courses with an international component 
Joint degree programs 


Students’ participation in international studies 


Research Internationally cooperative research programs 
Internationally focused research centres 
International researchers 


Internationally acknowledged research achievements 


Engagement International network and partnership 


International presence of alumni 


The identified common components of university internationalisation in previous studies indicate the 
structural and philosophical similarities in a variety of institutional practices about internationalisation. 
These generic values and strategies allow meaningful comparisons to be made between universities in 
relation to internationalisation. Although considerable efforts have been made to develop measures for 
internationalisation, little has been known about how internationalisation performance is assessed in 
individual universities. In other word, whether these established instruments have been applied to help 
university leaders to monitor their performance in terms of internationalisation has yet been explored. 


Research questions 


In order to fill the gap identified above, this study intended to investigate how internationalisation is 
currently being monitored and measured in universities. Specifically, two questions were developed to 
steer the investigation: 


1. In what way is university internationalisation performance being measured in individual 


universities? 
2. What are the difficulties in measuring university internationalisation performance? 


Methodology 
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Field interviews were employed to collect the data for this study. In-depth interviews are flexible and 
adaptable to individual situations (Davies, 1997). They provide actual words of participants, offering 
many different perspectives on the topic being studied. During the interview, open-ended questions are 
used so that interviewees can best voice their experiences unconstrained by any perspective of the 
researcher or past research findings (Creswell, 2012). They permit probing into the context of, the 
reasons for, and answers to questions about beliefs and concerns. The interviews were semi-structured 
in nature. A purposive sampling was adopted to identify the interviewees. By using this sampling 
method, “particular setting and persons are deliberately selected for the important information they can 
provide that cannot be gotten as well from other choices” (Maxwell, 1996, p. 87). The final sample of 
interviewees consisted of 11 Pro-Vice Chancellors (PVCs) of international and 4 Directors of 
International Offices from 15 elite research universities in Australia and China (9 universities in 
Australia and 6 in China). To fully protect the participants’ confidentiality, the names of universities 
will be referred to by a pseudonym in any publications. Elite research universities were selected 
because they are the most active and powerful players in the process of internationalisation compared 
to other types of universities (Geuna, 1998). Universities in China and Australia were selected because 
they provide distinctive national contexts to examine the phenomenon. And they differ in terms of 
status towards internationalisation and the focus of international strategy. The distinctions between 
them enrich the perspectives on the phenomenon being studied. 


Interviews were conducted in the native language of the interviewees because their responses would 
be fuller and more nuanced in native language. Language is after all a tool for constructing reality 
(Spradley, 1979, p. 17). The length of the interview was flexible. The longest lasted 2 hours; the 
shortest was 45 minutes. All the interviews were audiotaped and the recordings were transcribed. 
Thematic analysis (Creswell, 2012) was used to analyse the qualitative data collected through 
interviews. The data were scrutinised and coded to the identified main themes. Generally, the thematic 
analysis was a classification system to provide insights regarding the frequency and patterns of factors 
that affect the phenomenon of interest. 


Importance of measuring university internationalisation performance 


The findings of the study show that among the 15 sample universities, 8 universities are using specific 
measurements to monitor internationalisation performance. Another five have statistics in relation to 
internationalisation to various degrees. However, the records do not serve the purpose of assessment or 
informing policy and practice. There is no assessment for internationalisation in the other two 
universities. They show interest in better monitoring performance; however, the interest has yet been 
transformed into practice. Irrespective of the stage the universities have reached with regard to 
measuring internationalisation, almost all of the interviewed university policy makers confirmed the 
importance of having measurements of internationalisation to position oneself and inform evidence- 
based decision-making. One of the PVCs commented: 


We use these evidence-based data to inform the approach to our global strategy. For 
example, it is very powerful if I want to change the funding support for outbound mobility. 
By looking at the profile of all the students going abroad, we will say that a differential 
monetary support should be given to students from disadvantaged backgrounds because 
they need to be given that opportunity, you know, access. The data of who is doing what, 
where and why give me the strategic levers to shift strategy. But I can’t do without data 
(Interview A, Australia). 


Another PVC approved of this view, saying that: 


We spent a lot of time on tracing recruitment because they’re drivers of every university. I 
also would like to know what the companies are thing, what the governments are thinking 
and then we could fulfil them. We measure everything that we could measure. We’re trying 
to get more management information, which can help us make decisions (Interview G, 
Australia). 


Defining measures for internationalisation would certainly help to guide activities in more established 
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universities. It could also be of great value to universities that have yet to find their place in the world. 
As one interviewee suggested, ‘Indonesia or Vietnam or Cambodia government will pay for 
consultancy for telling them what their three hundred new universities should be doing in terms of 
monitoring internationalisation (Interview B, Australia). Moreover, the information provided by 
measurements could be used by universities to identify the problems in their current practice in 
relation to internationalisation, as illustrated in the statement that ‘the data help us identify the pattern 
of our international practice, which show that it is biased to advanced science and technology country 
partners. It is biased to English speaking countries (Interview A, China). 


How is internationalisation performance being measured in universities? 


In universities that developed measurements or statistics for internationalisation, two approaches to 
conduct the assessment are evident. One is the quantitative dominated approach, which uses 
performance indicators as the main instrument for collecting data. The other is a mix of qualitative and 
quantitative approaches. In this approach, both indicators and surveys are employed to gather 
information in relation to different aspects of internationalisation performance. Different indicators 
have been developed by universities as proxies for internationalisation achievements. In one of the 
sample universities, 13 indicators were built to measure international collaboration. They are student 
exchange, study abroad, course work, and recruitment (for volume and range of source countries) and 
sponsorship. Graduate students, sponsorship, joint PhDs, alumni from partners are used to measure the 
activities of engagement. With regard to research, joint publications and joint projects are the 
indicators (Interview A, Australia). Other institutions may employ a different set of indicators. For 
example, as one PVC stated: 


On monitoring, we always start with the international students, which are monitored on a 
daily basis. We look at how many students had applied. Where are they coming from? In 
terms of study abroad we monitor that by counting how many students were sent out. I 
don’t pay attention to how many are coming in since I don’t really care. What I care about 
is how many we get out. And I look at that probably three times a year because we send 
them out in big adventures. About international research collaboration, we have got two 
variables as our benchmarks. One is percentage of co-publications with non-Australian 
publishers and the other is amount of research funds raised by non-Australian sources. I’m 
also interested in number of MOUs, number of research agreements and I also keep an eye 
on some really big projects we have (Interview I, Australia). 


In most universities in China, assessing internationalisation performance is part of the annual 
education evaluation. Every year the central office of the university evaluates the academic 
performance of schools and departments, and internationalisation forms different proportions of the 
overall evaluation. Several indicators are collected to generate an overall score of a school’s 
internationalisation performance. The indicators usually include international conference, outbound 
mobility of faculties, visiting scholars, student exchange, co-publications, joint courses and joint 
degree programs (Interview C & D, China). 


In two other universities in China, the situation is slightly different as both of the Directors mentioned 
internationalisation is not being measured in their institutions. However, they have established 
relatively comprehensive statistics of international activities. These descriptive data are collected 
annually and reported to the President. Instead of serving an informative purpose or being used for 
guiding the target setting, the information is only regarded as a summary of their work in relation to 
internationalisation (Interview B & E, China). 


Indicators are powerful in gathering information about institutional internationalisation performance, 
particularly of components that can be easily quantified and captured by numbers. For elements that 
deny quantification, qualitative information needs to be collected through surveys or program reviews, 
as illustrated by this statement: 


We conduct reviews of all units involved with international activities. The reviews are 
looking at how appropriately the units are structured and how appropriate their policies are 
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to contribute to the university internationalisation. That review is not saying that you only 
have 5,000 international students and that is bad. It is saying do you have effective 
mechanism for communicating with schools to help them recruit students and the answer is 
no. (Interview B, Australia). 


This approach is adopted in other universities. In one investigated institution, a range of systemic 
review reports about internationalisation is generated annually. There is input from all the schools and 
all the central divisions on their internationalisation strategies, and their comments on the international 
plan. A panel, consisting of 8-10 people, digests and discusses the information. The chair of the panel 
produces the report that will go to the senior executive (Interview F, Australia). 


In exploring the experience and satisfactions of students and staff, surveys are employed to obtain 
feedback. ‘Staff surveys are used to assess staff responses to our institutional strategies to see whether 
people believe in our institutional strategies and are supportive of it (Interview E, Australia).’ 


Other than the established measurements of internationalisation, interviewees were also asked to 
identify the measures of other components they are looking for. They expressed their interest in 
capturing information about short-term outbound mobility of academic staff, appropriate measures for 
internationalising students’ experience and the way to capture the work they do overseas in capacity 
building countries which is funded by different bodies. 


Difficulties in measuring university internationalisation 


Although various instruments have been developed to help visualise and measure university 
internationalisation, barriers that impede the establishment of effective assessment were highlighted by 
the interviewees. One major obstacle in measuring internationalisation is the availability of the data, 
particularly in universities in Australia, where the management system is more decentralised. Most 
international activities like research collaborations and faculty exchanges take place in individual 
schools and departments. It is a challenge for university central offices to trace and record those 
activities. This challenge is demonstrated in the statement below: 


Because the operation of the university has been so decentralised, I think there are lots of 
happens without the central university knowing about them. You will hear one day that a 
senior minister of the government visits one of our schools. And I can easily get angry with 
that. There is nobody tells me that there is a minister on the campus from a foreign country. 
But in a way that the facts happen in the first place means that the connection the school 
has or the reputation of the school is so high quality that the minister would travel there and 
visits the school without university knowing about it (Interview B, Australia). 


In addition, some interviewees showed their concerns about the appropriateness and quality of the 
proxies of internationalisation. Because of the nature and characteristics of some disciplines, indicators 
may not be high-quality or meaningful proxies for the real performance of internationalisation in a 
given institution. For example, the number of international publications has been commonly used as a 
measure; however, it tells us very little about the internationalisation of research where that the 
university has researchers who are involved in mass physics projects like CERN or in physics 
publication with over 200 authors. They produce hundreds of papers a year, but there is not necessarily 
any deep collaboration. For those multi-billion dollars research projects, anyone involved in any way 
becomes the author on the paper. In this regard, weight has to be applied to almost all the multi- 
authored papers or the number should be refined in discipline analysis before it can be used to draw 
any meaningful conclusion about the internationalisation of research of a university (Interview F, 
Australia). 


Even the indicator is a qualified proxy for internationalisation the terminology can be problematic. A 
straightforward indicator can indeed be understood and calculated in a variety of ways. The variations 
or ambiguities in the wording of some indicators are a cause for concerns. If the data were produced 
by different definitions, they would result in unreliable, and perhaps, quite erroneous conclusions 
about university internationalisation, particularly in making comparisons between institutions. As one 
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of the interviewed directors stated, international faculty is a tricky concept. It can be defined by 
nationality, by the primary language used by the staff or by the place of birth. In the case of China, 
whether the foreign citizens of Chinese origin should be taken into account is also a problem 
(Interview C, China). 


A Director from China shared his understanding of the reason why there are no measurements of 
internationalisation implemented in his institution. He argued that, because of the complexity of 
internationalisation and the various factors exert the influence on university internationalisation 
performance, even with the data it remains puzzling to diagnose the responsible party for the problem, 
as illustrated in the statement below: 


Even you have the measures and when you look at your data, and you look at your 
performance, it is very difficult to figure out who should be blamed for or who should get 
the credit. For example, the number of international visiting scholars is determined, to some 
extent, by the financial resources the university can provide. The outbound mobility of 
students is another example. It also subjects to the availability of financial resources as well 
as the quota that the university can get from some national schemes, which is beyond the 
control of the university (Interview F, China). 


Discussion: measuring internationalisation performance, the urgent challenges 
university leaders encounter 
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